The present invention relates to a method of playing a set of multimedia applications.
Such a method may be used in, for example, a computer via a browser providing access to MPEG audio and video data over an internet-type distribution system.
An audio-video player is a program running on a computer that decodes audio and video streams in order to produce an audio-visual presentation. FIG. 1 is a block diagram of a method of playing audio and video frames in accordance with the prior art. Said method plays MPEG-4 data and comprises a demultiplexing step (DEMUX) for splitting an MPEG-4 encoded data stream (IS) into an audio stream (AS) and several video streams (VS1 to VSn). Such a method comprises three main tasks.
It firstly comprises an audio decoding and rendering task (DR). This task decodes an audio stream (AS) and drives the sound rendering system by providing decoded audio samples to sound system hardware. The sound system hardware converts these digital audio samples into an analog sound signal (SO), which is sent to loudspeakers (LS).
It also comprises a video decoding task (DEC). This task decodes at least one video stream (VS) and stores the decoded video frames in a video frame buffer (BUF).
Finally, it comprises a video rendering task (REN). This task takes the decoded video frames (VF) from the video frame buffer and supplies pixels corresponding to the decoded video frames to video system hardware in order to compose a video scene (SC). The video rendering step also performs all the video frame conversions which are necessary to drive a monitor (MON).
The European patent application no0 817 103 describes a method and system for providing access to multimedia content data on an internet-type distribution system. Such a method allows to play multimedia data according to the above description, using a web browser, the audio and video playback being performed on separate threads of control.
It is an object of the invention to provide a method of playing a set of multimedia applications, which allows a better synchronization and real-time playing of audio and video frames. The present invention takes the following aspect into consideration.
In the method according to the prior art, each multimedia application contains its own scheduler, and runs it in its own thread. So, even if all application-intrinsic tasks are synchronized, the overall application is not necessarily synchronized. In the example of the integration of a multimedia player in a web browser, each application has its own scheduler and the multimedia player can decode and give to the browser a ready-to-show frame faster than the browser can possibly display. But there is no need to decode 25 frames of a video sequence per second if the browser can display only 10 frames per second. In the same manner, the multimedia player does not need to render at 25 frames per second if the central processing unit, hereinafter referred to as CPU, cannot handle the decoding of more than 10 frames per second.
To overcome the limitations of the prior art, the method of playing a set of multimedia applications in accordance with the invention is characterized in that it comprises the steps of:
creating a common scheduler at a start time in order to provide a target time,
registering the tasks into the common scheduler, and
controlling the execution of the tasks as a function of the target time.
The use of a common scheduler, contrary to generic scheduling strategies such as the ones implemented in operating system kernels, allows the overall application to remain synchronized while having real time playing.
Moreover, the method of playing a set of multimedia applications in accordance with the invention is characterized in that it further comprises a step of computing a local time for a task from the target time provided by the scheduler and timing information associated with said task.
Such a method ensures a correct operation of the overall application. For that purpose, three specific embodiments are proposed.
In the first one, the method of playing a set of multimedia applications is characterized in that the computing step comprises the sub-steps of determining a start time offset associated with a task from a difference between a start time of the task and the scheduler start time, and calculating the local time from a difference between the target time and the start time offset. For example, when a video sequence is being played, a video decoding task should need to know the target time in order to decode the correct frame. However, the decoding task will first try to decode the frame corresponding to the target time, and will not succeed in this, because the target time is not zero seconds. That is why the method in accordance with the invention computes a local time for this task, thus ensuring a proper video decoding in the context of a common scheduler.
In the second embodiment, the method of playing a set of multimedia applications is characterized in that the task is comprised in a multimedia application coming from an external source, and the computing step comprises the sub-steps of determining an external time offset of the external multimedia application from a difference between the scheduler start time and a start time of the external multimedia application, and calculating the local time from a sum of the target time and the external time offset. As a consequence, the method allows digital encoded data streams to be read from local storage or to be received from a broadcast or network.
In the third embodiment, the method of playing a set of multimedia applications is characterized in that the task is applied to a time limited data stream and the computing step comprises the sub-steps of determining a duration of the data stream, incrementing a counter each time the data stream is finished, and calculating the local time from a difference between the target time and a product of the duration and the counter. As a consequence, the local time varies from zero to the duration of the data stream and the method allows to loop on said data stream.
These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.